Compile

This topic is about how to compile Doris on the ARM64 platform.

Note that this is for reference only. Other errors may occur when compiling in different environments.

KylinOS Version：

CPU Model:

This method works for Doris versions after commit 7f3564.

Download ldb_toolchain_gen.aarch64.sh

For detailed instructions, please refer to Compile with ldb-toolchain

Note that you need to download the corresponding aarch64 versions of jdk and nodejs:

Create root directories

Git

JDK8 (2 methods)

Maven

NodeJS

ldb-toolchain

Configure environment variables

Install other environments and components

Update apt-get repository

Check the shell command set

The Ubuntu shell installs dash instead of bash by default. It needs to be switched to bash for proper execution. Run the following command to view the details of sh and confirm which program corresponds to the shell:

The shell can be switched back to bash by:

Then select no to confirm.

After these steps, dash will no longer be the default shell tool.

Create root directories

Git

JDK8

Maven

NodeJS

ldb-toolchain

Configure environment variables

Install other environments and components

If there is data returned, that means AVX2 is supported; otherwise, AVX2 is not supported.

File not found when compiling the third-party library libhdfs3.a.

Problem Description

During the compilation and installation process, the following error occurrs:

not found lib/libhdfs3.a file or directory

Cause

The third-party library dependency is improperly downloaded.

Solution

Use a third-party download repository

REPOSITORY_URL contains all third-party library source packages and their historical versions.

python command not found

Problem Description

An exception is thrown when executing build.sh

/opt/doris/env.sh: line 46: python: command not found

Python 2.7.18

Cause

The system uses python2.7, python3.6, python2, python3 by default to execute python commands. Doris only requires python 2.7+ to install dependencies, so you just need to add a command python to connect.

Solution

Establish a soft link to the python command in \usr\bin

There is no output directory after compilation

Problem Description

Cause

Compilation fails. Try again.

Solution

spark-dpp compilation fails

Problem Description

After compiling build.sh, compiling to Spark-DPP fails with an error

Failed to execute goal on project spark-dpp

Cause

This error message is caused by download failure (connection to the repo.maven.apache.org central repository fails).

Could not transfer artifact org.apache.spark:spark-sql_2.12:jar:2.4.6 from/to central (https://repo.maven.apache.org/maven2): Transfer failed for https://repo .maven.apache.org/maven2/org/apache/spark/spark-sql_2.12/2.4.6/spark-sql_2.12-2.4.6.jar: Unknown host repo.maven.apache.org

Solution

No space left, compilation fails

Problem Description

Failed to build CXX object during compilation, error message showing no space left

fatal error: error writing to /tmp/ccKn4nPK.s: No space left on device 1112 | } // namespace doris::vectorized compilation terminated.

Cause

Insufficient free space on the device

Solution

Expand the free space on the device by deleting files you don’t need, etc.

Failed to start FE, transaction error -20

Problem Description

When starting FE, a transaction error 20 is reported with UNKNOWN status.

[BDBEnvironment.setup():198] error to open replicated environment. will exit. com.sleepycat.je.rep.ReplicaWriteException: (JE 18.3.12) Problem closing transaction 20. The current state is:UNKNOWN. The node transitioned to this state at:Fri Apr 22 12:48:08 CST 2022

Cause

Insufficient hard disk space

Solution

Free up hard disk space or mount a new hard disk

Abnormal BDB environment setting, disk search error

Problem Description

An exception is reported when starting FE after migrating the drive letter where FE is located

2022-04-22 16:21:44,092 ERROR (MASTER 172.28.7.231_9010_1650606822109(-1)|1) [BDBJEJournal.open():306] catch an exception when setup bdb environment. will exit. com.sleepycat.je.DiskLimitException: (JE 18.3.12) Disk usage is not within je.maxDisk or je.freeDisk limits and write operations are prohibited: maxDiskLimit=0 freeDiskLimit=5,368,709,120 adjustedMaxDiskLimit=0 maxDiskOverage=0 freeDiskShortage=1,536,552,960 diskFreeSpace =3,832,156,160 availableLogSize=-1,536,552,960 totalLogSize=4,665 activeLogSize=4,665 reservedLogSize=0 protectedLogSize=0 protectedLogSizeMap={}

Cause

FE has been migrated to another location, which doesn’t match the hard disk information stored in the metadata; or the hard disk is damaged or not mounted

Solution

Could not find pkg.m4 file in pkg.config

Problem Description

A “file not found” error occurrs during compilation:

Couldn’t find pkg.m4 from pkg-config. Install the appropriate package for your distribution or set ACLOCAL_PATH to the directory containing pkg.m4.

Cause

There is something wrong with the compilation of the third-party library libxml2 .

Possible Reasons:

Solution

Copy the pkg.m4 file in the ldb/aclocal directory into the libxml2/m4 directory, and recompile the third-party library.

Failed to execute test CURL_HAS_TLS_PROXY

Problem Description

An error is reported during the compilation process of the third-party package:

-- Performing Test CURL_HAS_TLS_PROXY - Failed CMake Error at cmake/dependencies.cmake:15 (get_property): INTERFACE_LIBRARY targets may only have whitelisted properties. The property “LINK_LIBRARIES_ALL” is not allowed.

The log shows: curl No such file or directory

fatal error: curl/curl.h: No such file or directory 2 | #include compilation terminated. ninja: build stopped: subcommand failed.

Cause

There is an error in the compilation environment. The gcc is of version 9.3.0 that comes with the system, so it is not compiled with ldb, so you need to configure the ldb environment variable.

Solution

Configure ldb environment variables

Other problems

Problem Description

The follow error prompts are all due to one root cause.

Cause

Not compiled with ldb-toolschain

Solution